Measurement Issues in the Design of State Accountability Systems

We examined the practices, policies, and procedures used in all 50 states for evaluating school and school district effectiveness. Our primary interest was the study of methodological and measurement issues in the collection, analysis, and reporting of information for accountability purposes. Accountability occupies a central role in current educational reform efforts nationally. 1999), " ...accountability is the watchword, and policymakers are busy looking for ways to reward success and punish failure in an effort to improve public education " (p.3). Most states are now engaged in the development of new assessment and accountability systems that can measure student, school and district performance. Forty-eight states now require some form of student testing (Bond, Roeber, & Connealy, 1997). Many states are in the process of revamping testing systems and instruments, most commonly with an emphasis on " criterion-referenced " and " standards-based " assessment instruments. Thirty-six states now publish some form of annual report cards on individual school performance. A number of states have already passed or are considering legislation providing monetary incentives for " high-performing " schools and sanctions for " low-performing " schools. This push for accountability reflects widespread interest on the part of the public, legislators, policy makers, and parents in obtaining information on school performance and effectiveness. There is an increasingly high-stakes environment surrounding the measurement and assessment of many complex issues involved in the development and implementation of these systems that require careful scrutiny and there are a number of cautions being raised about the systems being movement is to be successful, more careful attention to the design and development of accountability systems will be needed. One aspect that is critical to that development is the judicious consideration of how information is gathered, analyzed, interpreted and reported. There are a number of challenges in attempting to measure school performance and effectiveness. The school environment is an exceedingly complex milieu in which many forces are State Accountability Systems 4 simultaneously at work impacting the school participants. There are several kinds of "effects" on student outcomes. There are characteristics of the students themselves that have to do with their background, environment, and characteristics. There are also contextual and environmental influences on students; for example, it is clear that students with fewer resources and less opportunity have lower absolute levels of achievement. These kinds of variables must be considered in developing accountability systems. First, it is important to discover whether …